Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Proposal: Support Sector and Industry data from Markets Tab #2058

Merged
merged 1 commit into from
Sep 22, 2024
Merged

Feature Proposal: Support Sector and Industry data from Markets Tab #2058

merged 1 commit into from
Sep 22, 2024

Conversation

ericpien
Copy link
Contributor

@ericpien ericpien commented Sep 18, 2024 •
edited
Loading

The following data is available on: https://finance.yahoo.com/sectors/:

  • sector / industry overview
  • sector / industry top companies
  • sector / industry research reports
  • sector top ETFs
  • sector top Mutual Funds
  • industries within sector
  • industry's sector
  • industry's top performing companies
  • industry's top growth companies

Resolves:

Sample code:

import yfinance as yf
tech = yf.Sector('technology')
software = yf.Industry('software-infrastructure')

# Common information
tech.key
tech.name
tech.symbol
tech.ticker
tech.overview
tech.top_companies
tech.research_reports

# Sector information
tech.top_etfs
tech.top_mutual_funds
tech.industries

# Industry information
software.sector_key
software.sector_name
software.top_performing_companies
software.top_growth_companies

# Ticker to Sector and Industry
msft = yf.Ticker('MSFT')
tech = yf.Sector(msft.info.get('sectorKey'))
software = yf.Industry(msft.info.get('industryKey'))

# Sector and Industry to Ticker
tech_ticker = tech.ticker
tech_ticker.info
software_ticker = software.ticker
software_ticker.history()

@ericpien ericpien marked this pull request as ready for review September 18, 2024 15:24
@sshcli
Copy link

sshcli commented Sep 18, 2024

Thank you @ericpien 🙌

@ValueRaider
Copy link
Collaborator

I ran tech.symbol a few times, not many, and triggered a messy error:

ERROR    Failed to get sector data for 'technology' reason: 401 Client Error: Unauthorized for url: https://query1.finance.yahoo.com/v1/finance/sectors/technology?formatted=true&withReturns=true&lang=en-US&region=US&crumb=4WOcuFKaIqF
DEBUG    Got response: 
DEBUG    -------------
Traceback (most recent call last):
  File "/home/test/yfinance-ericpien/yfinance/domain/sector.py", line 53, in _fetch_and_parse
    result = self._fetch(self._query_url, self.proxy)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/test/yfinance-ericpien/yfinance/domain/domain.py", line 59, in _fetch
    result = self._data.get_raw_json(query_url, user_agent_headers=self._data.user_agent_headers, params=params_dict, proxy=proxy)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/test/yfinance-ericpien/yfinance/data.py", line 399, in get_raw_json
    response.raise_for_status()
  File "/home/test/.local/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://query1.finance.yahoo.com/v1/finance/sectors/technology?formatted=true&withReturns=true&lang=en-US&region=US&crumb=4WOcuFKaIqF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/test/yfinance-ericpien/./test.py", line 12, in <module>
    print(tech.symbol)
          ^^^^^^^^^^^
  File "/home/test/yfinance-ericpien/yfinance/domain/domain.py", line 34, in symbol
    self._ensure_fetched(self._symbol)
  File "/home/test/yfinance-ericpien/yfinance/domain/domain.py", line 97, in _ensure_fetched
    self._fetch_and_parse()
  File "/home/test/yfinance-ericpien/yfinance/domain/sector.py", line 66, in _fetch_and_parse
    logger.debug(f" {data}")
                     ^^^^
UnboundLocalError: cannot access local variable 'data' where it is not associated with a value

Also, you can put hardcoded stuff like SECTOR_INDUSTY_MAPPING in const.py, keep the logic clean.

@ericpien
Copy link
Contributor Author

@ValueRaider thanks for reviewing. I made the changes for const.py and the UnboundedLocalError portion of the log you shared.

I can't reproduce the HTTPError portion of the log. I ran the code below successfully which rules out the error being related to "running it a few times". Can you try again?

for s in const.SECTOR_INDUSTY_MAPPING:
    sector = yf.Sector(s)
    print(f'sector symbol: {sector.symbol}')
    for i in const.SECTOR_INDUSTY_MAPPING[s]:
        industry = yf.Industry(i)
        print(f'industry symbol: {industry.symbol}')

@ValueRaider
Copy link
Collaborator

Your change has improved the error - it still happens randomly but is cleaner. Probably related to my spam, I'm doing other fetches.

One more error:

tech = yf.Sector('technology')
tech.research_reports

  File "yfinance/domain/domain.py", line 54, in research_reports
    self._ensure_fetched(self._research_reports)
                         ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Sector' object has no attribute '_research_reports'. Did you mean: '_research_report'?

@ericpien
Implement Sector and Industry
3a0b6c2 
Sector and Industry class to query the data available on: https://finance.yahoo.com/sectors/:
- sector / industry overview
- sector / industry top companies
- sector / industry research reports
- sector top ETFs
- sector top Mutual Funds
- industries within sector
- industry's sector
- industry's top performing companies
- industry's top growth companies
@ericpien
Copy link
Contributor Author

Thanks. Corrected the param name and checked the other params are correct as well.

@ValueRaider ValueRaider merged commit 1890ff2 into ranaroussi:dev Sep 22, 2024
1 check passed
@ValueRaider
Copy link
Collaborator

I'm hoping to release in a week, I'm just finishing testing improvements to dividend repair.

This was referenced Sep 23, 2024
Open
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ericpien @sshcli @ValueRaider